Driving-operation assist and recording medium

ABSTRACT

A vehicle-operation assist includes a circumferential-state imager for imaging a circumferential state of a vehicle with a camera and generating a circumferential-state image; a synthetic-image generator for generating a synthetic image by superimposing on the circumferential-state image, an assumed-movement pattern of the vehicle performing a predetermined series of driving operations; and a display for displaying the synthetic image. The circumferential-state imager has at least one camera and a camera parameter table for storing characteristics of the camera and generating the circumferential-state image on the basis of the camera characteristics.

THIS APPLICATION IS A U.S. NATIONAL PHASE APPLICATION OF PCTINTERNATIONAL APPLICATION PCT/JP99/05509.

TECHNICAL FIELD

The present invention relates to a driving-operation assist forassisting driving operations of a vehicle and a recording medium forstoring a program for making a computer execute all or some of functionsof each means of the driving-operation assist.

BACKGROUND ART

A conventional general driving-operation assist estimates a moving traceof a vehicle corresponding to a steering angle of a steering wheel forbackward movement of the vehicle by a steering sensor for detecting thesteering angle of the steering wheel. When a vehicle moves backward, animage of a rear or side-rear field of view photographed by a camera isdisplayed and moreover, when the steering wheel is operated, a movingtrace of the vehicle estimated correspondingly to the steering angle ofthe steering wheel is superimposed on the image of the rear or side-rearfield of view. According to the assist, driving operations by a driverwill be performed as described below. That is, a driver moves a vehicleto a place where the vehicle can be probably parked while fixing thesteering wheel of the vehicle. Then, at the place, the driver finds asteering angle capable of moving the vehicle to a space for parking thevehicle without operating any steering wheel while confirming avehicle-moving trace estimated by operating the steering wheel. Then, bymoving the vehicle backward toward a parking space while keeping thesteering angle, parking is theoretically completed.

A conventional example of the above driving-operation assist isdisclosed in the official gazette of Japanese Patent Laid-Open No.1-14700.

However, to park a vehicle by using the above assist, it is necessary tofind a place where the vehicle can be moved to a parking space and thendetermine a steering angle at which the steering wheel is fixed. Tomaster these operations, skill is necessary. Moreover, when the size ofa vehicle to be driven is changed, a sense differs. Therefore, thedriving know-how accumulated during skill is not greatly useful.

Incidentally, to park a vehicle, it is generally difficult to completeparking operations while keeping a steering angle of a steering wheelconstant from start of the parking operations except a case in whichthere is no obstacle nearby. For example, to perform parallel parking, adriver first moves a vehicle backward by turning the steering wheel in aproper direction while moving the vehicle to a place for parking thevehicle from a parking-operation start position and then, moves thevehicle to a target place by inversely turning the steering wheel whenthe vehicle properly moves backward. That is, in case of taking parallelparking as an example, it is difficult to park a vehicle while keeping asteering angle of a steering wheel constant.

Moreover, as for a conventional assist, if a driver slightly moves thesteering wheel of a vehicle, a moving route of the vehicle re-estimatedby the slight steering-angle change is displayed. Therefore, the drivermay be confused.

That is, as for a conventional driving-operation assist, a driver cannotintuitively find a place where a vehicle can be easily moved to aparking space at a glance while confirming an image obtained bysynthesizing the parking space, a state around the space, and a routefor guiding the vehicle to the parking space.

DISCLOSURE OF THE INVENTION

The present invention is made to solve the above problems of anconventional driving-operation assist and its object is to provide adriving-operation assist by which a driver can directly confirm therelation between movements of a vehicle according to a predeterminedseries of driving operations and the circumferential state by displayingmovements of the vehicle when the driver performs the predeterminedseries of driving operations together with the circumferential state andthe load of the driver can be reduced.

Furthermore, it is another object of the present invention to provide arecording medium for storing a program for making a computer execute allor some of functions of each means of the above driving-operationassist.

To solve the above described problems the present invention provides adriving-operation assist characterized by comprising:

circumferential-state imaging means for imaging a circumferential stateof a vehicle with a camera and generating a circumferential-state imageand/or storing the generated circumferential-state image;

synthetic-image generating means for generating a synthetic image bysuperimposing on the circumferential-state image, an assumed-movementpattern which is the video data showing movement of the vehicle in caseof performing a predetermined series of driving operations for thevehicle; and

displaying means for displaying the synthetic image.

Another aspect of the present invention (corresponding to claim 26[sic;claim 33]) is a recording medium characterized by storing a program formaking a computer execute all or some of functions of each means of theabove driving-operation assist.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a configuration of a driving-operationassist of a first embodiment of the present invention;

FIG. 2 shows a top view and an elevation view of a vehicle provided withcameras of an imaging section 101 of a driving-operation assist of thefirst embodiment of the present invention;

FIG. 3 is an elevation view showing a viewpoint of a virtual camera ofthe driving-operation assist of the first embodiment of the presentinvention;

FIG. 4 is an illustration showing a circumferential-state image viewedfrom the virtual camera of the driving-operation assist of the firstembodiment of the present invention;

FIGS. 5( a) and 5(b) are illustrations showing assumed-movement data ofthe driving-operation assist of the first embodiment of the presentinvention;

FIGS. 6( a) and 6(b) are illustrations showing assumed-movement data ofthe driving-operation assist of the first embodiment of the presentinvention;

FIG. 7 is an illustration showing a synthetic image of thedriving-operation assist of the first embodiment of the presentinvention;

FIG. 8 is an illustration showing a synthetic image of thedriving-operation assist of the first embodiment of the presentinvention;

FIG. 9 is an illustration showing movement of a vehicle at the time ofperforming parallel parking to the left side;

FIGS. 10( a) to 10(c) are illustrations showing synthetic images of thedriving-operation assist of the first embodiment of the presentinvention;

FIG. 11 shows illustrations of variation of assumed-movement patternsstored in assumed-movement-pattern storing means 108 of thedriving-operation assist of the first embodiment of the presentinvention;

FIG. 12 is an illustration showing a modification of the synthetic imageof the driving-operation assist of the first embodiment of the presentinvention;

FIG. 13 is a block diagram showing a configuration of adriving-operation assist of a second embodiment of the presentinvention;

FIGS. 14( a) to 14(c) are illustrations showing synthetic images of thedriving-operation assist of the second embodiment of the presentinvention;

FIG. 15 is a block diagram showing a configuration of adriving-operation assist of a third embodiment of the present invention;

FIGS. 16( a) to 16(c) are illustrations showing synthetic images of thedriving-operation assist of the third embodiment of the presentinvention;

FIG. 17 is a block diagram showing a configuration of adriving-operation assist of a fourth embodiment of the presentinvention;

FIG. 18 is a block diagram showing a configuration of adriving-operation assist of a fifth embodiment of the present invention;

FIGS. 19( a) to 19(d) are illustrations showing synthetic images of thedriving-operation assist of the fifth embodiment of the presentinvention;

FIG. 20 is a block diagram showing a configuration of adriving-operation assist of a sixth embodiment of the present invention;

FIGS. 21( a) to 21(e) are illustrations showing synthetic images of thedriving-operation assist of the sixth embodiment of the presentinvention;

FIG. 22 is a block diagram of a configuration of a modification of thedriving-operation assist of the sixth embodiment of the presentinvention;

FIG. 23 is a block diagram showing a configuration of adriving-operation assist of a seventh embodiment of the presentinvention;

FIG. 24 is a conceptual view showing a mapping table stored in a mappingtable 2302 of the driving-operation assist of the seventh embodiment ofthe present invention;

FIG. 25 is a block diagram showing a configuration of adriving-operation assist of an eighth embodiment of the presentinvention;

FIGS. 26( a) to 26(c) are illustrations showing synthetic images of thedriving-operation assist of the eighth embodiment of the presentinvention;

FIG. 27 is a block diagram showing a configuration of adriving-operation assist of a ninth embodiment of the present invention;

FIGS. 28( a) to 28(c) are illustrations showing synthetic images of thedriving-operation assist of the ninth embodiment of the presentinvention;

FIG. 29 is a block diagram showing a configuration of adriving-operation assist of a tenth embodiment of the present invention;

FIGS. 30( a) and 30(b) are illustrations showing synthetic images of thedriving-operation assist of the tenth embodiment of the presentinvention;

FIG. 31 is a block diagram showing a configuration of adriving-operation assist of an eleventh embodiment of the presentinvention;

FIGS. 32( a) and 32(b) are illustrations showing synthetic images of thedriving-operation assist of the eleventh embodiment of the presentinvention;

FIG. 33 is an illustration showing a synthetic image of thedriving-operation assist of the eleventh embodiment of the presentinvention;

FIG. 34 is an illustration showing a synthetic image of thedriving-operation assist of the eleventh embodiment of the presentinvention;

FIG. 35 is an illustration showing a synthetic image of thedriving-operation assist of the eleventh embodiment of the presentinvention;

FIG. 36 is a graph for explaining a contact-hazard evaluation functionof the driving-operation assist of the eleventh embodiment of thepresent invention;

FIG. 37 is an illustration showing a synthetic image of thedriving-operation assist of the eleventh embodiment of the presentinvention;

FIG. 38 is an illustration [sic; expression] showing a contact-hazardevaluation function of the driving-operation assist of the eleventhembodiment of the present invention;

FIG. 39 shows examples of assumed movement data of the driving-operationassist of the eleventh embodiment of the present invention;

FIG. 40 is an illustration showing a synthetic image of thedriving-operation assist of the eleventh embodiment of the presentinvention;

FIG. 41 is an illustration showing a synthetic image of thedriving-operation assist of the eleventh embodiment of the presentinvention;

FIG. 42 is a block diagram showing a configuration of adriving-operation assist of a twelfth embodiment of the presentinvention;

FIG. 43 is an illustration showing a synthetic image of thedriving-operation assist of the twelfth embodiment of the presentinvention;

FIG. 44 is a block diagram showing a configuration of thedriving-operation assist of the twelfth embodiment of the presentinvention;

FIG. 45 is an illustration showing a synthetic image of thedriving-operation assist of the twelfth embodiment of the presentinvention; and

FIG. 46 is an illustration showing a synthetic image of thedriving-operation assist of the twelfth embodiment of the presentinvention.

DESCRIPTION OF SYMBOLS

-   101 Imaging section-   102 Superimposing means-   103 Camera parameter table-   104 Space reconfiguring means-   105 Space-data buffer-   106 Viewpoint converting means-   107 Displaying means-   108 Assumed-movement-pattern storing means-   1301, 1501 Start detecting means-   1302, 1703 Integrating means-   1303, 1503, 1704 Space converting means-   1502 Image tracking means-   1701 Start inputting means-   1702 Driving controlling means-   1801 Trace correcting means-   2001 CG-image synthesizing means.-   2201 Camera-   2301 Mapping means-   2302 Mapping table-   2501 Final-position inputting means-   2502 Start-position determining means-   2503 Space fixing means

BEST MODE FOR CARRYING OUT THE INVENTION

Embodiments of the present invention will be described by referring tothe accompanying drawing.

First Embodiment

First, a first embodiment of the present invention will be described byreferring to the drawings.

FIG. 1 is a block diagram showing a configuration of thedriving-operation assist of this embodiment. The driving-operationassist of this embodiment is mainly used to assist driving operationsfor garaging or parallel parking and the like.

As shown in FIG. 1, the driving-operation assist of this embodiment isconfigured by an imaging section 101 comprising N cameras (camera 1 tocamera N), a camera parameter table 103 for storing camera parameterswhich are characteristics of the cameras, space configuring means 104for generating space data relating each pixel configuring an imageoutput from each camera to a point in a three-dimensional space inaccordance with a camera parameter, viewpoint converting means 106 forgenerating an image viewed from a predetermined viewpoint as acircumferential-state image by referring to the space data, a space-databuffer 105 for temporarily storing the space data,assumed-movement-pattern storing means 108 for storing assumed-movementdata including assumed-movement patterns, superimposing means 102 forsuperimposing the assumed-movement pattern on the circumferential-stateimage and generating a synthetic image, and displaying means 107 fordisplaying a synthetic image.

An assembly including the imaging section 101, camera parameter table103, space reconfiguring means 104, and viewpoint converting means 106corresponds to circumferential-state imaging means of the presentinvention and the superimposing means 102 corresponds to synthetic-imagegenerating means of the present invention.

First, a detailed configuration of the imaging section 101 and aprocedure in which a circumferential-state image of the presentinvention is generated in accordance with video data imaged by theimaging section 101 will be described below.

FIG. 2 shows a top view and an elevation view of a vehicle provided withcameras of the imaging section 101. In this example, six cameras 201 to206 are arranged on the roof portion of a vehicle as N=6. The sixcameras 201 to 206 are arranged so that a part of each camera overlapswith a part of the imaging range of other camera and no dead angle isproduced flatly.

The camera parameter table 103 stores camera parameters of each of theabove cameras (parameters showing camera characteristics such as settingposition, setting angle, lens-strain correction value, and focaldistance of camera). The space reconfiguring means 104 generates spacedata relating each pixel configuring an image output from each camera inaccordance with the camera parameters to a point in a three-dimensionalspace on the basis of a vehicle. The space-data buffer 105 temporarilystores the above space data and the viewpoint converting means 106generates an image viewed from an optional viewpoint such as a viewpointof a virtual camera 301 shown in FIG. 3 as a circumferential-state imageby referring to space data and thereby synthesizing pixels.

FIG. 4 shows a circumferential-state image viewed from the viewpoint ofthe virtual camera 301 shown in FIG. 3. This example shows a case ofperforming parallel parking, in which two vehicles currently parked areshown on the circumferential-state image as an obstacle 401 and anobstacle 402.

Then, a procedure will be described in which the superimposing means 102generates a synthetic image of the present invention and finally, thedisplaying means 107 displays the synthetic image.

The assumed-movement-pattern storing means 108 stores anassumed-movement pattern which is video data showing movement of avehicle to which a typical vehicle driving operation is applied andtime-series data showing the relation between vehicle moving distance(moving distance due to rotation of tire) and steering angle of steeringwheel (rotation angle of steering wheel) as assumed-movement data of thepresent invention.

Among stored assumed-movement data, the assumed-movement data forperforming the driving operation of parallel parking to the left side isshown in FIG. 5 and the assumed-movement data for performing the drivingoperation for garaging to the right side is shown in FIG. 6. FIGS. 5( a)and 6(a) respectively show an assumed-movement pattern which is videodata corresponding to a driving operation according to the time-seriesdata shown in FIGS. 5( b) and 6(b) and showing operation start positions501 and 601 (corresponding to an assumed-movement start area of thepresent invention), operation end positions 502 and 602 (correspondingto an assumed-movement end area of the present invention), and tiretraces 503 and 603 (corresponding to video data showing vehicle tiretraces of the present invention).

First, a driver selects one of the assumed-movement patterns stored inthe assumed-movement-pattern storing means 108 by pattern selectingmeans (not illustrated). The superimposing means 102 superimposes aselected assumed-movement pattern (for example, FIG. 5( a)) on acircumferential-state image generated by the viewpoint converting means106 (for example, FIG. 4) to synthesize them and generate a syntheticimage of the present invention, and the displaying means 107 displaysthe synthetic image. In this case, for example, by adjusting theoperation start position 501 in FIG. 5( a) to the current position ofthe vehicle, the operation end position 502 becomes an operation endposition when starting a driving operation corresponding to theassumed-movement pattern from the current position, that is, a parkingposition.

FIG. 7 shows a synthetic image obtained by synthesizing theassumed-movement patterns shown in FIG. 5 and FIG. 8 shows a syntheticimage obtained by synthesizing the assumed-movement patterns shown inFIG. 6.

That is, in FIG. 7 (FIG. 8), the driver can perform parallel parking tothe left side (garaging to the right side) by moving the vehicle to astart position 701 (801) where the obstacles 401 and 402 (803 and 804)do not interfere with a parking position 702 (802), the tire trace 503(603), and the start position 701 (801) and starting a series of drivingoperations according to time-series data from the position so as to parkthe vehicle at the parking position 702 (802).

A detailed procedure for moving the vehicle to the start position 701(801) will be described below by using a case of performing parallelparking to the left side as an example and referring to FIG. 9 and FIGS.10( a) to 10(c).

FIG. 9 is an illustration showing movement of a vehicle when performingparallel parking to the left side. As shown in FIG. 9, to park thevehicle at a target parking position 902, a driver of the vehicle mustmove the vehicle located at a current position 901 to a target startposition 903 by assuming the operation start position 501 when adjustingthe operation end position 502 of the assumed-movement pattern (FIG. 5(a)) for performing parallel parking to the left side to the targetparking position 902 as the target start position 903.

The relative positional relation between the operation end position 502and the operation start position 501 in FIG. 5( a) corresponds to a caseof performing a driving operation in accordance with the time-seriesdata in FIG. 5( b). As for an actual driving operation, fine adjustmentcan be made in accordance with fine adjustment of a currently-operatedsteering wheel.

When the driving-operation assist of this embodiment is not used, thedriver must move the vehicle to the target start position 903 byassuming the obstacles 401 and 402 and the target parking position 902in accordance with a scene which can be confirmed from the inside of thevehicle through direct viewing or a mirror. In this case, there is aproblem that an operation for the driver to assume the obstacles 401 and402 and the target parking position 902 in accordance with the scenewhich can be confirmed from the inside of the vehicle through directviewing or a mirror requires skill. Moreover, even when a size of avehicle or a position of a mirror is changed, there is a problem that adriver cannot easily correspond to the change.

However, when the driving-operation assist of this embodiment is used,the assumed-movement pattern shown in FIG. 5( a) is superimposed on thecircumferential-state image viewed from a viewpoint of a virtual cameraas if being imaged from a position just above the vehicle as shown inFIG. 4 and thereby, the synthetic image shown in FIG. 7 is generated anddisplayed for a driver.

Therefore, at the time of performing a driving operation for moving thevehicle to the target start position 903 in FIG. 9, the operation endposition 502 when assuming the current position 901 as the operationstart position 501 is displayed as a parking position 1001 correspondingto the current position together with tire traces by displayingadjusting the current position 901 of the vehicle to the operation startposition 501 in FIG. 5( a) as shown in FIGS. 10( a) to 10(c). When thevehicle is located at the current position 901 where the parkingposition 1001 coincides with the target parking position 902, movementto the target start position 903 is completed.

That is, if a parking operation is started from the position when thesynthetic image 1 in FIG. 10( a) is displayed on the displaying means107, the parking position 1001 overlaps with the obstacle 402.Therefore, a driver can confirm at a glance that the parking operationmust be started from a position where the vehicle is further movedforward (upward in FIG. 10( a)).

Moreover, if a parking operation is started from the position when thesynthetic image 2 in FIG. 10( b) is displayed on the displaying means107, a tire trace overlaps with the obstacle 401. Therefore, the drivercan confirm at a glance that the parking operation must be started froma position where the vehicle is further moved backward (downward in FIG.10( a)).

Furthermore, if a parking operation is started from the position whenthe synthetic image 3 in FIG. 10( c) is displayed on the displayingmeans 107, a tire trace does not overlap with the obstacle 401 or 402and the driver can confirm at a glance that the parking position 1001 isa position suitable for parking. Therefore, it is possible to confirmthat a parking operation can be started from the position.

Thus, by virtually synthesizing images showing the positional relationbetween a vehicle viewed from a position just above the vehicle,circumferential obstacles, parking end position, and tire traces andshowing a synthesized image to a driver, the driver can directly confirmthe positional relation between them at a glance. As a result, thedriver can confirm a place suitable for start of a parking operation ata glance, easily move a vehicle to the place, and start a parkingoperation. Therefore, it is possible to park the vehicle at a purposedposition more safely and accurately.

Assumed-movement-pattern-operation start and end positions and a tiretrace are intrinsic to each vehicle. For example, they are greatlydifferent in small vehicle and large vehicle. This can be settled bystoring an assumed-movement pattern in the assumed-movement-patternstoring means 108 in FIG. 1 every vehicle. Therefore, a driver canperform driving operations while confirming the relation between anassumed-movement pattern corresponding to the vehicle and acircumferential obstacle even if the vehicle is changed to another one.

Moreover, when vehicles are changed, it is considered that positions ofand the number of vehicle-mounted cameras shown in FIG. 2 are alsochanged. However, this can also be settled by the fact that the cameraparameter table 103 in FIG. 1 stores camera parameters of each cameraevery vehicle and an image shown to a driver is not directly influenced.Therefore, the driver can perform driving operations while confirmingthe relation between an assumed-movement pattern corresponding to avehicle displayed in a synthetic image almost similarly displayed beforevehicles are changed and a circumferential obstacle even if vehicles arechanged and thereby, camera positions are changed.

As described above, according to this embodiment, it is possible todirectly confirm a driving vehicle, an obstacle, and a purposed positionat a glance in a driving operation such as parallel parkingconventionally requiring a considerable skill for a driver, perform moresafe and accurate driving operation, and greatly decrease an operationload of the driver.

Moreover, a driver can perform a driving operation while confirming therelation between an assumed-movement pattern corresponding to a vehicleand a circumferential obstacle or the like, almost similarly to a statebefore vehicles are changed even if vehicles are changed and thereby, itis possible to greatly decrease a load of a driver's skill for change ofvehicles.

FIG. 11 shows a variation of assumed-movement patterns stored in theassumed-movement-pattern storing means 108 in FIG. 1. The patterns areright and left parallel-parking assumed-movement patterns 1101 and 1102and right and left garaging assumed-movement patterns 1103 and 1104. Asdescribed above, a driver selects any one of these patterns by patternselecting means (not illustrated). Areas to be displayed as syntheticimages are also determined as shown by outer frames of theassumed-movement patterns 1101 to 1104 in FIG. 11 correspondingly to theabove four assumed-movement patterns. That is, an operation startposition is assumed as a current vehicle position and a rectangular areaincluding a tire trace and an operation end position is assumed as asynthetic-image area.

Though a vehicle is not imaged from a vehicle-mounted camera in general,it is permitted in this case to hold CG data for a vehicle andactual-vehicle data and display these data in a synthetic image bysuperimposing them on each other similarly to trace data.

For this embodiment, assumed-movement patterns of the present inventionare described as video data showing an operation start position(assumed-movement start area of the present invention), an operation endposition (assumed-movement end area of the present invention), and tiretraces (image data showing vehicle tire traces of the presentinvention). However, other pattern is also permitted, for example, itcan be that traces (video data showing a vehicle moving area of thepresent invention) in which a projection of a vehicle moves are includedinstead of or together with tire traces. In short, it is only necessarythat assumed-movement patterns of the present invention serve as videodata showing movement of a vehicle when assuming that a predeterminedseries of driving operations are previously applied to the vehicle.

Moreover, as described above, if directly displaying a tire trace and/ora vehicle moving area, there is no margin for contact between thevehicle and an obstacle or the like. Therefore, as shown in FIG. 12, itis also permitted to display a margin line 1201 set outward by apredetermined value (e.g. 50 cm) from a tire trace or the outer marginof a vehicle moving area.

Furthermore, for this embodiment, it is described that acircumferential-state image of the present invention is obtained bysynthesizing video data imaged by the imaging section 101 in real time.However, at a place where the same driving operation is frequentlyperformed and circumferential state is hardly fluctuated, other patternis also permitted, for example, it can be to store the data produced inaccordance with already-imaged video data in the space-data buffer 105and use the data.

Second Embodiment

Then, a second embodiment of the present invention will be describedbelow by referring to the accompanying drawings.

FIG. 13 is a block diagram showing a configuration of thedriving-operation assist of the second embodiment of the presentinvention. The driving-operation assist of this embodiment is alsomainly used to assist driving operations such as garaging and parallelparking. Therefore, in this embodiment, it is assumed that a componentwith no explanation is the same as the case of the first embodiment anda component provided with the same symbol as the case of the firstembodiment has the function as the case of the first embodiment unlessotherwise explained. Moreover, it is assumed that each modified exampledescribed for the first embodiment can be applied to this embodiment byapplying the same modification unless otherwise described.

As shown in FIG. 13, the configuration of the driving-operation assistof this embodiment is different from that of the driving-operationassist of the first embodiment in that start detecting means 1301,integrating means 1302, and space converting means 1303 are included.

The start detecting means 1301 to which a gear signal showing forwardmovement or backward movement and a steering-wheel-turning-angle signalshowing a front-wheel steering angle are input judges that a drivingoperation (parking operation) corresponding to an assumed-movementpattern stored in the assumed-movement-pattern storing means 108 isstarted when a gear signal shows a backward-movement state and afront-wheel steering angle according to a steering-wheel-turning-anglesignal reaches a certain value or more, which corresponds to operationstart detecting means of the present invention.

The integrating means 1302 integrates the steering-wheel rotation angleand the number of revolutions of rear wheels and computes a spatialmovement change of a vehicle up to the current point of time after adriving operation (parking operation) is started, which corresponds tomoving-position computing means of the present invention.

The space converting means 1303 moves an assumed-movement pattern inaccordance with the spatial movement change and an assembly of thesuperimposing means 102 and space converting means 1303 corresponds tosynthetic-image generating means of the present invention.

In this embodiment, a procedure until a circumferential-state image ofthe present invention is generated in accordance with video data imagedby the imaging section 101 is the same as the procedure described forthe first embodiment. Moreover, a procedure until an actual drivingoperation corresponding to an assumed-movement pattern is started in aprocedure until the superimposing means 102 generates a synthetic imageof the present invention and the displaying means 107 displays thesynthetic image is the same as that described for the first embodiment.

A procedure until the superimposing means 102 generates a syntheticimage of the present invention and the displaying means 107 displays thesynthetic image after an actual driving operation corresponding to anassumed-movement pattern is started is described below by using a caseof performing parallel parking to the left side as an example.

When a driver starts a driving operation corresponding to parallelparking to the left side, a gear signal shows a backward-movement stateand a front-wheel steering angle according to asteering-wheel-turning-angle signal reaches a certain value or more.Therefore, the start detecting means 1301 judges that a drivingoperation (parking operation) corresponding to parallel parking to theleft side is started, communicates to the integrating means 1302 thatthe driving operation (parking operation) is started, and thereafterinputs the steering-wheel-rotation-angle signal and thenumber-of-revolution-of-rear-wheel signal to the integrating means 1302.

The integrating means 1302 integrates the inputsteering-wheel-rotation-angle signal andnumber-of-revolution-of-rear-wheel signal after start of the drivingoperation and computes the positional relation between a current vehicleposition 1402 and a parking-operation start position 1401 as shown inFIG. 14( a).

In accordance with the computed positional relation, the spaceconverting means 1303 moves an assumed-movement pattern 1403corresponding to parallel parking to the left side so that the operationstart position (501 in FIG. 5) coincides with the parking-operationstart position 1401 as shown in FIG. 14( b). In other words, the spaceconverting means 1303 spatially fixes the assumed-movement pattern 1403at a position when the parking operation starts after start of thedriving operation.

After start of a driving operation, the superimposing means 102synthesizes the assumed-movement pattern 1403 spatially fixed at theparking-operation start position and the current vehicle position 1402by superimposing the pattern 1403 and the position 1402 on acircumferential-state image to generate a synthetic image of the presentinvention as shown in FIG. 14( c). The displaying means 107 displays thesynthetic image. Because circumferential-state images such as theobstacles 401 and 402 are naturally fixed in a space, the positionalrelation between the circumferential-state images and theassumed-movement pattern 1403 is fixed in the synthetic image. Moreover,because a synthetic image is an image viewed from a viewpoint fixed onto a vehicle, only the current vehicle position 1402 is fixed on thescreen when the vehicle moves and circumferential-state images and theassumed-movement pattern 1403 are displayed while they relatively movein FIG. 14( c).

That is, according to this embodiment, a circumferential-state imageviewed from a viewpoint of a virtual camera moves in accordance withmovement of an actual vehicle and the superimposed and synthesizedassumed-movement pattern 1403 also moves in accordance with movement ofa vehicle computed by the integrating means 1302. Therefore, thecircumferential-state image and the assumed-movement pattern 1403 showcoincident movement. Because a driver only has to operate a steeringwheel along a tire trace of an assumed-movement pattern displayed ateach point of time, a simpler and safer vehicle operation is realized.

Third Embodiment

Then, a third embodiment of the present invention will be describedbelow by referring to the accompanying drawings.

FIG. 15 is a block diagram showing a configuration of thedriving-operation assist of the third embodiment of the presentinvention. The driving-operation assist of this embodiment is alsomainly used to assist driving operations for garaging, parallel parkingand the like. Therefore, in this embodiment, it is assumed that acomponent with no explanation is the same as the case of the firstembodiment and a component provided with the same symbol as the case ofthe first embodiment has the same function as the case of the firstembodiment unless otherwise explained. Moreover, each modified exampledescribed for the first embodiment can be applied to this embodiment byapplying the same modification unless otherwise described.

As shown in FIG. 15, the configuration of the driving-operation assistof this embodiment is different from that of the driving-operationassist of the first embodiment in that start detecting means 1501 andimage tracking means 1502, and space converting means 1503 are included.

The start detecting means 1501 has the same functions as the startdetecting means 1301 in FIG. 13 described for the second embodimentexcept that a steering-wheel-turning-angle signal or anumber-of-revolution-of-rear-wheel signal is not output to any otherunit.

The image tracking means 1502 stores positional information of the wholeor a part (e.g. operation end position) of video data forassumed-movement patterns on the basis of the whole or a part (e.g.obstacle) of video data for circumferential-state images on a syntheticimage when a driving operation (parking operation) is started andcorresponds to positional-information storing means of the presentinvention.

The space converting means 1503 moves an assumed-movement pattern inaccordance with the positional information and an assembly configured bythe superimposing means 102 and space converting means 1503 correspondsto synthetic-image generating means of the present invention.

In this embodiment, a procedure until a circumferential-state image ofthe present invention is generated in accordance with video data imagedby the imaging section 101 is the same as that described for the firstembodiment. Moreover, a procedure until an actual driving operationcorresponding to an assumed-movement pattern is started in a procedureuntil the superimposing means 102 generates a synthetic image of thepresent invention and the displaying means 107 displays the syntheticimage is the same as that described for the first embodiment.

A procedure until the superimposing means 102 generates a syntheticimage of the present invention and the displaying means 107 displays thesynthetic image after an actual driving operation corresponding to anassumed-movement pattern is started is described below by using a caseof performing parallel parking to the left side as an example.

When a driver starts a driving operation corresponding to parallelparking to the left side, a gear signal shows a backward-movement stateand a front-wheel steering angle according to asteering-wheel-turning-angle signal reaches a certain value or more.Therefore, the start detecting means 1501 judges that a drivingoperation (parking operation) corresponding to parallel parking to theleft side is started and communicates to the image tracking means 1502that the driving operation (parking operation) is started.

When the image tracking means 1502 receives the information showing thatthe driving operation (parking operation) is started, there is obtainedthe video data for an end-position circumferential image 1603 includinga part of the obstacle 402 which is a part of the video data for acircumferential-state image and a parking-operation end position 1602 onthe then synthetic image (FIG. 16( a)) through the space-data buffer105, and the video data is stored. After start of the driving operation,the means 1502 finds the portion concerned of the obstacle 402 from acircumferential-state image at each point of time (obtained through thespace-data buffer 105), adjusts the portion concerned of the obstacle402 in the end-position circumferential image 1603 to the portionconcerned of the obstacle 402 in a circumferential-state image, andthereby determines the positional relation between the parking-operationend position 1602 and the circumferential-state image at that point oftime. That is, the image tracking means 1502 tracks the positionalrelation between the parking-operation end position 1602 and thecircumferential-state image at each point of time.

According to the positional relation, the space converting means 1503moves an assumed-movement pattern corresponding to parallel parking tothe left side so that the operation end position (502 in FIG. 5)coincides with the parking-operation end position 1602. In other words,after start of the driving operation, the space converting means 1503spatially fixes the assumed-movement pattern to a position at start ofthe parking operation.

As shown in FIG. 16( c), after start of the driving operation, thesuperimposing means 102 superimposes the assumed-movement pattern 1605spatially-fixed to the position at start of the parking operation andthe current vehicle position 1604 on a circumferential-state image andsynthesizes them to generate a synthetic image of the present invention.The displaying means 107 displays the synthetic image. Becausecircumferential-state images of the obstacles 401 and 402 or the like,are naturally fixed in a space, the positional relation between thecircumferential-state image and the assumed-movement pattern 1403 isfixed in the synthetic image.

Moreover, in FIG. 16( c), because the synthetic image is an image viewedfrom a viewpoint fixed on a vehicle, if the vehicle moves, only thecurrent vehicle position 1604 is fixed on the screen and thecircumferential-state image and the assumed-movement pattern 1605 aredisplayed while they relatively move. That is, as for thedriving-operation assist of this embodiment, if a procedure is executedunder the same condition as the case of the driving-operation assist ofthe second embodiment, the synthetic image shown in FIG. 16( c) becomessame as the synthetic image shown in FIG. 14( c).

According to this embodiment, because a circumferential-state imageviewed from a viewpoint of a virtual camera moves in accordance withmovement of an actual vehicle and the superimposed and synthesizedassumed-movement pattern 1605 also moves in accordance with the movementof the vehicle. Therefore, both show coincident movement. Because adriver only has to operate a steering wheel along trace data for anassumed-movement pattern displayed at each point of time, a simpler andsafer vehicle operation is realized.

Fourth Embodiment

Then, a fourth embodiment of the present invention will be describedbelow by referring to the accompanying drawings.

FIG. 17 is a block diagram showing a configuration of thedriving-operation assist of the fourth embodiment of the presentinvention. The driving-operation assist of this embodiment is alsomainly used to assist driving operations for garaging, parallel parkingand the like. Therefore, it is assumed that a component with noexplanation is the same as the case of the first embodiment and acomponent provided with the same symbol as the case of the firstembodiment has the same function as the case of the first embodimentunless otherwise explained. Moreover, it is assumed that each modifiedexample described for the first embodiment can be applied to thisembodiment by applying the same modification unless otherwise described.

As shown in FIG. 17, the configuration of the driving-operation assistof this embodiment is different from that of the driving-operationassist of the first embodiment in that start inputting means 1701,driving controlling means 1702, integrating means 1703, and spaceconverting means 1704 are included.

The start inputting means 1701 is used for a driver to input anddesignate start of an actual driving operation (parking operation)corresponding to an assumed-movement pattern, which corresponds tooperation start detecting means of the present invention.

The driving controlling means 1702 automatically controls operations ofa vehicle by controlling a steering-wheel rotation angle and the numberof revolutions of rear wheels in accordance with time-series data (e.g.FIG. 5( b)) corresponding to an assumed-movement pattern when adriving-operation start instruction is input, which corresponds tooperation controlling means of the present invention.

The integrating means 1703 integrates a steering-wheel rotation angleand the number of revolutions of rear wheels and computes a spatialmovement change of a vehicle up to the current point of time after adriving operation (parking operation) is started, which corresponds tomoving-position computing means of the present invention. That is, theintegrating means 1703 has the same function as the integrating means1302 in FIG. 13 described for the second embodiment.

The space converting means 1704 moves an assumed-movement pattern inaccordance with the spatial movement change and an assembly constitutedof the superimposing means 102 and space converting means 1704corresponds to synthetic-image generating means of the presentinvention. That is, the space converting means 1704 has the samefunctions as the space converting means 1303 in FIG. 13 described forthe second embodiment.

In this embodiment, a procedure until a circumferential-state image ofthe present invention is generated in accordance with video data imagedby the imaging section 101 is the same as that described for the firstembodiment. Moreover, a procedure until an actual driving operationcorresponding to an assumed-movement pattern is started in a procedureuntil the superimposing means 102 generates a synthetic image of thepresent invention and the displaying means 107 displays the syntheticimage is the same as that described for the first embodiment.

A procedure until the superimposing means 102 generates a syntheticimage of the present invention and the displaying means 107 displays thesynthetic image after an actual driving operation corresponding to anassumed-movement pattern is started is described below by using a caseof performing parallel parking to the left side as an example.

A driver locates a vehicle at a position suitable for start of a parkingoperation while viewing a synthetic image displayed on the displayingmeans 107 before starting the parking operation and then, inputs aninstruction for start of the parking operation to the start inputtingmeans 1701. The start inputting means 1701 communicates that thedriving-operation start instruction is input to the driving controllingmeans 1702 and integrating means 1703 through theassumed-movement-pattern storing means 108.

When receiving a parking-operation start instruction, the drivingcontrolling means 1702 automatically controls driving of a vehicle bygenerating a steering-wheel-rotation-angle control signal and anumber-of-revolution-of-rear-wheel control signal in accordance with thetime-series data (FIG. 5( b)) corresponding to an assumed-movementpattern and controlling a steering-wheel control system and a rear-wheelcontrol system.

In case of receiving a parking-operation start instruction, theintegrating means 1703 integrates a steering-wheel-operation-anglesignal and a number-of-revolution-of-rear-wheel signal to compute thepositional relation between the current vehicle position 1402 and theparking-operation start position 1401 as shown in FIG. 14( a).

As shown in FIG. 14( b), the space converting means 1704 moves theassumed-movement pattern 1403 corresponding to parallel parking to theleft side so that the operation start position (501 in FIG. 5) and theparking-operation start position 1401 coincide with each other inaccordance with the computed positional relation. In other words, afterstart of a driving operation, the space converting means 1704 spatiallyfixes the assumed-movement pattern 1403 to the parking-operation startposition.

A subsequent procedure in which the superimposing means 102 generates asynthetic image and the displaying means 107 displays the syntheticimage is the same as that described for the second embodiment. When thevehicle reaches a predetermined parking position, the driving controlmeans 1702 stops the vehicle in accordance with the time-series data andthereby, the parking operation is completed.

According to this embodiment, it is possible to obtain an advantage thata steering-wheel operation, and the like are automatically performed inaddition to the advantages described for the second embodiment afterstart of a driving operation is designated. Because a driver only has toconfirm that a steering-wheel operation is automatically generated inaccordance with the trace data for a displayed assumed-movement patternat each point of time and monitor that a new obstacle appears, a simplerand safer vehicle operation is realized.

Fifth Embodiment

Then, a fifth embodiment of the present invention will be describedbelow by referring to the accompanying drawings.

FIG. 18 is a block diagram showing a configuration of thedriving-operation assist of the fifth embodiment of the presentinvention. The driving-operation assist of this embodiment is alsomainly used to assist driving operations for garaging, parallel parkingand the like. Therefore, it is assumed that a component with noexplanation is the same as the case of the first embodiment and acomponent provided with the same symbol as the case of the firstembodiment has the same function as the case of the first embodimentunless otherwise explained. Moreover, each modified example describedfor the first embodiment can be applied to this embodiment by applyingthe same modification unless otherwise described.

As shown in FIG. 18, the configuration of the driving-operation assistof this embodiment is different from that of the first embodiment inthat trace correcting means 1801 is included.

The trace correcting means 1801 corrects an assumed-movement pattern andtime-series data in accordance with a driving-operation start positionand an operation end position input from a driver, which corresponds topattern correcting means of the present invention.

In this embodiment, a procedure until a circumferential-state image ofthe present invention is generated in accordance with video data imagedby the imaging section 101 is the same as that described for the firstembodiment. Moreover, a procedure until an assumed-movement patternstored in the assumed-movement-pattern storing means 108 is displayed ona synthetic image by making the operation start position of the patterncoincide with the current vehicle position in a procedure until thesuperimposing means 102 generates a synthetic image of the presentinvention and the displaying means 107 displays the synthetic image isthe same as that described for the first embodiment.

A procedure until a driver corrects an assumed-movement pattern andtime-series data by using the trace correcting means 1801 and theassumed-movement pattern and time-series data are displayed on asynthetic image after the assumed-movement pattern is displayed on thesynthetic image is described below by using a case of performinggaraging to the left side as an example.

As shown in FIG. 9( a), a case is assumed in which as a result of movinga vehicle to a current position 1901 where anassumed-movement-pattern-operation end position coincides with a targetparking position 1902 in order to perform a garaging operation byassuming the target parking position 1902 as an operation end positionso that a driver does not contact with an obstacle 1904 or 1905, it isfound that a tire trace 1903 of the assumed-movement pattern interfereswith the obstacle 1905.

When the assumed-movement-pattern storing means 108 stores anassumed-movement pattern for a garaging operation to another left side,it is possible to study whether a parking operation is smoothlyperformed by selecting the assumed-movement pattern stored in the means108 by pattern selecting means (not illustrated). However, when themeans 108 does not store the assumed-movement pattern or when anotherassumed-movement pattern also interferes with an obstacle, a drivercorrects the assumed-movement pattern.

First, the driver moves a pattern showing a vehicle present at thecurrent position 1901 of a synthetic image (FIG. 19( a)) displayed onthe displaying means 107 to a new operation start position 1906 byinputting of a numerical value, a pointer, or other means as shown inFIG. 19( b).

When the new operation start position 1906 is determined, the tracecorrecting means 1801 obtains a new tire trace 1907 (refer to FIG. 19(c)) for a vehicle to move from the new operation start position 1906 tothe target parking position 1902 and generates new assumed-movementpatterns and time-series data corresponding to the new assumed-movementpatterns.

The superimposing means 102 adjusts an operation start position of thenew assumed-movement pattern to the current position 1901 of a vehicleand generates a synthetic image as shown in FIG. 19( d) and thedisplaying means 107 displays the synthetic image.

Therefore, a driver can park a vehicle to the target parking position1902 by moving the vehicle so as to adjust an operation end position1908 of the new assumed-movement pattern to the target parking position1902 and then, starting a driving operation (parking operation) inaccordance with the new assumed-movement pattern.

It is permitted to store the generated new assumed-movement pattern andtime-series data in the assumed-movement-pattern storing means 108 afterupdating an original assumed-movement pattern or to additionally storethe new assumed-movement pattern and time-series data in theassumed-movement pattern storing means 108 without updating the originalassumed-movement pattern. Moreover, it is permitted not to store them astemporary matters assumed-movement pattern and time-series data in themeans 108. Furthermore, it is permitted that a driver selects renewedstorage, additional storage, or non-storage each time.

Moreover, for this embodiment, it is described that an assumed-movementpattern to be renewedly stored or additionally stored in theassumed-movement-pattern storing means 108 is automatically obtained inaccordance with positions of a vehicle at start and end of driving inputby a driver. However, it is also permitted to perform an actual drivingoperation, sample time-series data at that time for the steering angleof a steering wheel, the number of revolution of wheels and the like,and generate and store an assumed-movement pattern in accordance withthe time-series data.

According to this embodiment, it is possible to realize an extensibledriving-operation assist compared to the driving-operation assist of thefirst embodiment.

Sixth Embodiment

Then, a sixth embodiment of the present invention will be describedbelow by referring to the accompanying drawings.

FIG. 20 is a block diagram showing a configuration of thedriving-operation assist of the sixth embodiment of the presentinvention. The driving-operation assist of this embodiment is alsomainly used to assist driving operations for garaging, parallel parkingand the like. Therefore, in this embodiment, it is assumed that acomponent with no explanation is the same as the case of the firstembodiment and a component provided with the same symbol as the case ofthe first embodiment has the same function as the case of the firstembodiment unless otherwise explained. Moreover, each modified exampledescribed for the first embodiment can be applied to this embodiment byapplying the same modification unless otherwise described.

As shown in FIG. 20, the configuration of the driving-operation assistof this embodiment is different from that of the driving-operationassist of the first embodiment in that CG-image synthesizing means 2001is included.

The CG-image synthesizing means 2001 stores three-dimensional datacorresponding to the assumed-movement patterns stored in theassumed-movement-pattern storing means 108 and generates athree-dimensional (or two-dimensional) image adjusted to a viewpoint ofa circumferential-state image, which corresponds to a part of thefunction of assumed-movement-pattern storing means of the presentinvention and a part of the function of synthetic-image generating meansof the present invention.

It is assumed that the viewpoint converting means 106 can changepositions of a viewpoint automatically or in accordance with an inputfrom a driver.

Moreover, as shown in FIG. 21( a), the assumed-movement-pattern storingmeans 108 stores positions of a plurality of virtual poles2001[sic;2101] arranged on the tire traces 503 together with storedassumed-movement patterns (operation start position 501, operation endposition 502, and tire traces 503). The CG-image synthesizing means 2001previously generates and stores three-dimensional-data (refer to FIG.21( b)) corresponding to the assumed-movement patterns in accordancewith the data for the assumed-movement patterns and the virtual poles2001[sic;2101].

In this embodiment, a procedure until a circumferential-state image ofthe present invention is generated in accordance with the video dataimaged by the imaging section 101 is the same as that described for thefirst embodiment. Moreover, a procedure until an actual drivingoperation corresponding to an assumed-movement pattern is started in aprocedure until the superimposing means 102 generates a synthetic imageof the present invention and the displaying means 107 displays thesynthetic image is the same as that described for the first embodiment.

A procedure until the superimposing means 102 generates a syntheticimage of the present invention and the displaying means 107 displays thesynthetic image after an actual driving operation corresponding to anassumed-movement pattern is started is described below by using a caseof performing parallel parking to the left side as an example.

Before starting an actual driving operation, a driver changes aviewpoint of a circumferential-state image used by the viewpointconverting means 106 to a viewpoint for viewing the rear from the backof the vehicle concerned from a position just above the vehicleconcerned. Or, when the viewpoint converting means 106 detects that anactual driving operation corresponding to an assumed-movement pattern isstarted, the viewpoint is automatically changed to the latter viewpoint.Specific means for detecting that an actual driving operation is startedincludes, for examples, means same as the start detecting means 1301described for the second embodiment.

Because viewpoints are changed, a circumferential-state image which isan output from the viewpoint converting means 106 becomes the imageshown in FIG. 21( c). The CG-image synthesizing means 2001 generates aCG image viewed from a viewpoint same as the viewpoint used by theviewpoint converting means 106 by adjusting the current position of thevehicle concerned to the operation start position 501. In this case, theCG image becomes the image shown in FIG. 21( d).

The superimposing means 102 superimposes the CG image on acircumferential-state image and synthesizes them to generate a syntheticimage of the present invention as shown in FIG. 21( e). The displayingmeans 107 displays the synthetic image. Because the synthetic image isan image viewed from a viewpoint fixed on a vehicle, the whole image isdisplayed while moving relatively when the vehicle moves in case of FIG.21( e).

In this embodiment, because a driver can determine a parking startposition while viewing the displayed image and confirming the relationbetween a virtual pole or an operation end position and an actualobstacle at a glance, the driver can perform a safe and secure drivingoperation.

In the above explanation, the CG-image synthesizing means 2001 generatesa CG image from a viewpoint same as the viewpoint used by the viewpointconverting means 106 in real time. However, it is also permitted to usea configuration of previously generating a CG image viewed from apredetermined viewpoint position every assumed-movement pattern andstoring these CG images.

Moreover, in this embodiment, it is described that acircumferential-state image viewed from a virtual camera is generatedsimilarly to the case of the first embodiment. However, when purposingonly an image viewing the rear from the back of a vehicle withoutchanging any viewpoint, it is also permitted to directly use an imageimaged by a camera set to the viewpoint position concerned as acircumferential-state image. A configuration of a driving-operationassist in this case is shown by the block diagram in FIG. 22. That is,the CG-image synthesizing means 2001 generates a CG image by obtainingthe data for a viewpoint of a vehicle-mounted camera 2201 from thecamera parameter table 103.

Seventh Embodiment

Then, a seventh embodiment of the present invention will be describedbelow by referring to the accompanying drawings.

FIG. 23 is a block diagram showing a configuration of thedriving-operation assist of the seventh embodiment of the presentinvention. The driving-operation assist of this embodiment is alsomainly used to assist driving operations for garaging, parallel parkingand the like. Therefore, in this embodiment, it is assumed that acomponent with no explanation is the same as the case of the firstembodiment and a component provided with the same symbol as the case ofthe first embodiment has the same function as the case of the firstembodiment unless otherwise explained. Moreover, it is assumed that eachmodified example described for the first embodiment can be applied tothis embodiment by applying the same modification unless otherwisedescribed.

As shown in FIG. 23, the configuration of the driving-operation assistof this embodiment is different from that of the driving-operationassist of the first embodiment in that mapping means 2301 and a mappingtable 2302 are included.

The mapping means 2301 fast performs the processing for converting animage input from each camera of the imaging section 101 into an imageviewed from an optional viewpoint.

The mapping table 2302 stores the data used for the mapping means 2301to perform conversion.

FIG. 24 is a conceptual view showing are example of a mapping tablestored in the mapping table 2302. The mapping table is configured bycells equal to the number of pixels of a screen (that is, a syntheticimage generated by the superimposing means 102) displayed by thedisplaying means 107. That is, the mapping table is configured so thatthe number of horizontal pixels on a display screen becomes equal to thenumber of columns of the table and the number of vertical pixels on thedisplay screen becomes equal to the number of rows of the table.Moreover, each cell has a camera number and pixel coordinates of animage photographed by each camera as data.

For example, the top-left cell (cell in which the data (1, 10, 10) isentered) in FIG. 24 shows the top left, that is, the portion of (0,0) onthe display screen and the mapping means 2301 performs the processing ofdisplaying the data for the pixel (10,10) of an image photographed by acamera No. 1 on a display screen (0,0)” in accordance with the datacontent (1,10,10) stored in the cell. By using the table, it isunnecessary to perform the computation for deciding the data for a pixelof a camera with which each pixel on a display screen should be replacedwhether to replace the data for a pixel of a replaced one. Therefore, itis possible to realize to accelerate the processing.

When the viewpoint converting means 106 uses a plurality of viewpointsby changing them, it is necessary that the table shown in FIG. 24 isstored in the mapping table 2302 every viewpoint.

Eighth Embodiment

Then, an eighth embodiment of the present invention will be describedbelow by referring to the accompanying drawings.

FIG. 25 is a block diagram showing a configuration of thedriving-operation assist of the eighth embodiment of the presentinvention. The driving-operation assist of this embodiment is alsomainly used to assist driving operations for garaging, parallel parkingand the like. Therefore, in this embodiment, it is assumed that acomponent with no explanation is the same as the case of the firstembodiment and a component provided with the same symbol as the case ofthe first embodiment has the same function as the case of the firstembodiment unless otherwise explained. Moreover, it is assumed that eachmodified example described for the first embodiment can be applied tothis embodiment by applying the same modification unless otherwisedescribed.

As shown in FIG. 25, the configuration of the driving-operation assistof this embodiment is different from that of the driving-operationassist of the first embodiment in that final-position inputting means2501, start-position determining means 2502, and space fixing means 2503are included.

The final-position inputting means 2501 inputs a target end position ofa driving operation by a pointer. It is also permitted to input thetarget end position by inputting a numerical value or through othermeans.

The start-position determining means 2502 obtains a start position of adriving operation corresponding to a target end position input by thefinal-position inputting means 2501 in accordance with anassumed-movement pattern corresponding to the driving operationconcerned.

The space fixing means 2503 makes the target end position coincide withthe operation end position and thereafter, spatially fixes anassumed-movement pattern corresponding to the driving operation, whichhas the functions of the integrating means 1302 and space convertingmeans 1303 in FIG. 13 (in FIG. 25, a rear-wheel-rotation-signal inputand a steering-wheel-turning-angle signal input are not illustrated).

Moreover, it is permitted that the means 2503 has the functions of theimage tracking means 1502 and space converting means 1503 in FIG. 15. Inthis case, however, it is necessary to receive an input of space datafrom the space buffer 105 [sic; space data buffer 105] similarly to thecase of the image tracking means 1502 in FIG. 15. An assembly configuredby the superimposing means 102 and space fixing means 2503 correspondsto synthetic-image generating means of the present invention.

In this embodiment, a procedure until a circumferential-state image ofthe present invention is generated in accordance with the video dataimaged by the imaging section 101 is the same as that described for thefirst embodiment. Moreover, a procedure until an assumed-movementpattern stored in the assumed-movement-pattern storing means 108 isdisplayed on a synthetic image by adjusting an operation start positionof the pattern to a current position of a vehicle in a procedure untilthe superimposing means 102 generates a synthetic image of the presentinvention and the displaying means 107 displays the synthetic image isthe same as that described for the first embodiment.

A procedure until a driver inputs a target end position of a drivingoperation by using the final-position inputting means 2501 and anassumed-movement pattern including a driving-operation start positioncorresponding to the target end position is displayed on the syntheticimage after an assumed-movement pattern is displayed on a syntheticimage will be described below by using a case of performing garaging tothe left side as an example.

As shown in FIG. 26( a), a case is assumed in which, as a result ofdriver's displaying a synthetic image of the present invention on thedisplaying means 107 to park a vehicle between the obstacles 401 and 402so as not to contact them, the parking position 1001 which is theoperation end position of the assumed-movement pattern 1403 using thecurrent position 901 of a vehicle as an operation start positionoverlaps with the obstacle 402.

The driver moves the parking position 1001 to a target position 2602 byusing a pointer 2601 displayed on the screen of the displaying means107. In this case, as shown in FIG. 26( b), the assumed-movement pattern1403 moves together with the parking position 1001. Therefore, theoperation start position of the assumed-movement pattern 1403 isdisplayed as a start position 2603 for starting a parking operation.

As shown in FIG. 26( c), even after the above moving operation iscompleted, the current position 901 of the vehicle is still displayed onthe screen of the displaying means 107. The driver only has to move thevehicle to the start position 2603 while viewing the screen. In thiscase, because the assumed-movement pattern 1403 is fixed in a space bythe space fixing means 2503, the relative positional relation betweenthe assumed-movement pattern 1403 and the obstacles 401 and 402 is notchanged.

According to this embodiment, because a driving-operation start positioncan be efficiently obtained in addition to the advantages described forthe first embodiment, it is possible to decrease the time required bystart of the operation.

Moreover, it is permitted to add a start-position guiding means to thedriving-operation assist of this embodiment, which computes a relativepositional relation with the current position 901 when the startposition 2603 is determined, obtains time-series data for asteering-wheel rotation angle and the number of revolutions of rearwheels necessary to guide a vehicle from the current position 901 up tothe start position 2603, generates a steering-wheel-rotation-anglecontrol signal and a number-of-revolution-of-rear-wheel control signalin accordance with the time-series data, controls a steering-wheelsystem and a rear-wheel driving system, and thereby automaticallycontrols driving of the vehicle and automatically guides the vehiclefrom the current position 901 up to the start position 2603. Thereby,because the vehicle is guided up to a start position without operationsby the driver, simpler and safer vehicle operations are realized.

Ninth Embodiment

Then, a ninth embodiment of the present invention will be describedbelow by referring to the accompanying drawings.

FIG. 27 is a block diagram showing a configuration of thedriving-operation assist of the ninth embodiment of the presentinvention. The driving-operation assist of this embodiment is differentfrom that of the second embodiment only in that outputs of the startdetecting means 1301 and the integrating means 1302 are input to theviewpoint converting means 106 and the viewpoint converting means 106changes viewpoints of a virtual camera in accordance with the outputs.

Therefore, in this embodiment, it is assumed that a component with noexplanation is the same as the case of the second embodiment and acomponent provided with the same symbol as the case of the secondembodiment has the same function as the case of the second embodimentunless otherwise explained. Moreover, it is assumed that each modifiedexample described for the second embodiment can be applied to thisembodiment by applying the same modification unless otherwise described.

In this embodiment, a procedure until a circumferential-state image ofthe present invention is generated in accordance with the video dataimaged by the imaging section 101 is the same as that described for thefirst embodiment. Moreover, a procedure until an actual drivingoperation corresponding to an assumed-movement pattern is started in aprocedure until the superimposing means 102 generates a synthetic imageof the present invention and the displaying means 107 displays thesynthetic image is the same as that described for the first embodiment.

A procedure until the superimposing means 102 generates a syntheticimage of the present invention and the displaying means 107 displays thesynthetic image after an actual driving operation corresponding to anassumed-movement pattern is started will be described below by using acase of performing parallel parking to the left side as an example.

Before an actual driving operation is started, a viewpoint position of avirtual camera is fixed to a position just above a vehicle as describedfor the first embodiment by referring to FIG. 10, the current vehicleposition 901 and the assumed-movement pattern 1403 using the position901 as an operation start position is fixed on a screen as shown in FIG.28( a), and circumferential-state images such as the obstacles 401, 402or the like, are displayed while relatively moving on the screen inaccordance with movement of the vehicle.

As shown in FIG. 28( b), when the current vehicle position 901 comes toa position corresponding to the target parking position 902, a driverstarts a driving operation corresponding to parallel parking to the leftside. When the driving operation is started, a gear signal shows abackward-movement state and a front-wheel steering angle according to asteering-wheel-turning-angle signal reaches a certain value or more.Therefore, the start detecting means 1301 judges that the drivingoperation (parking operation) corresponding to the parallel parking tothe left side is started and communicates that the driving operation(parking operation) is started to the integrating means 1302 andviewpoint converting means 106.

The integrating means 1302 integrates inputsteering-wheel-rotation-angle signal andnumber-of-revolution-of-rear-wheel signal after start of the drivingoperation to compute the positional relation between the current vehicleposition 1402 and the parking-operation start position 1401 as shown inFIG. 14( a).

In accordance with the computed positional relation, the spaceconverting means 1303 moves the assumed-movement pattern 1403corresponding to the parallel parking to the left side so that theoperation start position (501 in FIG. 5) of the pattern 1403 coincideswith the parking-operation start position 1401 as shown in FIG. 14( b).In other words, the space converting means 1303 spatially fixes theassumed-movement pattern 1403 to the position at start of the parkingoperation after start of the driving operation.

When the viewpoint converting means 106 receives that the drivingoperation (parking operation) is started, it fixes the then viewpointposition of the virtual camera to a space (ground). That is, after startof the driving operation, the circumferential-state images (obstacles401, 402 or the like) are fixed on the screen.

After start of the driving operation, the superimposing means 102superimposes the assumed-movement pattern 1403 spatially fixed to theposition at start of the parking operation and the current vehicleposition 1402 on the circumferential-state images and synthesizes themto generate a synthetic image of the present invention as shown in FIG.28( c). Because the viewpoint of the synthetic image is obtained byspatially fixing the viewpoint position of the virtual camera at startof the parking operation similarly to viewpoints ofcircumferential-state images. Therefore, the superimposing means 102generates the synthetic image by computing the positional relationcomputed by the integrating means 1302 inversely to a viewpoint. Thatis, in this synthetic image (FIG. 28( c)), the circumferential-stateimages (obstacles 401, 402 or the like) and the assumed-movement pattern1403 are fixed on the screen and the current vehicle position 1402 isdisplayed while relatively moving on the screen in accordance withactual movement of the vehicle.

According to this embodiment, because a viewpoint after start of aparking operation is fixed in a space, a driver can confirm a vehiclemoving state to a circumferential state of a parking space at a glance.

When a movement assumed in accordance with an assumed-movement patternincludes a wide range or is complex and a range larger than the field ofview of a camera of the imaging section 101 occurs after a drivingoperation is started, it is also permitted to display the range by usingthe data stored in the space-data buffer 105.

Tenth Embodiment

Then, a tenth embodiment of the present invention will be describedbelow by referring to the accompanying drawings.

FIG. 29 is a block diagram showing a configuration of thedriving-operation assist of the tenth embodiment of the presentinvention. The driving-operation assist of this embodiment is differentfrom that of the first embodiment only in that two cases are assumed inwhich a vehicle moves backward when turning a steering wheel up to themaximum angle clockwise and counterclockwise to tire rotation as defaultdriving patterns stored in the assumed-movement-pattern storing means108 as shown in FIG. 30( b), two circumscribed area traces 604 in aspace through which the whole of the vehicle passes in stead of the tiretraces 603 caused by assumed movements of the two cases are synthesizedby the superimposing means 102 at the same time as shown in FIG. 30( a),and the synthesized result is displayed on the displaying means 107.

Therefore, in this embodiment, it is assumed that a component with noexplanation is the same as the case of the first embodiment, a componentprovided with the same symbol as the case of the first embodiment hasthe same function as the case of the first embodiment unless otherwiseexplained. Moreover, it is assumed that each modified example describedfor the first embodiment can be applied to this embodiment by applyingthe same modification unless otherwise described.

According to this embodiment, two simplest cases are assumed in which avehicle moves backward when turning a steering wheel up to the maximumangle clockwise and counterclockwise as shown in FIG. 30( b) as defaultsof the assumed-movement storing pattern 108 [sic; assumed-movementpattern storing means]. Therefore, when the vehicle can be parked inaccordance with the defaults of the assumed-movement storing pattern 108[sic; assumed-movement pattern storing means], advantages can beobtained that it becomes unnecessary to change and select a plurality ofassumed-movement storing patterns and an operating load of a driver canbe reduced.

Moreover, it becomes unnecessary to change and select two left and rightassumed-movement storing patterns by simultaneously synthesizingcircumscribed area traces 604 produced due to assumed movements of theabove described two cases by the superimposing means 102 and bydisplaying the synthesized result on the displaying means 107, andconsequently there can be obtained such an effect that an operating loadof the driver can be reduced.

Furthermore, as shown in FIG. 30( a), the circumscribed area traces 60in a space through which the whole of the vehicle passes instead of thetire traces 603 are synthesized by the superimposing means 102, and thesynthesized result is displayed on the displaying means 107, and therebyan advantage can be obtained that a driver can easily and moreaccurately confirm whether a portion of a vehicle to be protruded beyonda tire trace such as the right front portion of the vehicle when parkingthe vehicle while turning left-backward contacts with an obstacle.

Eleventh Embodiment

Then, an eleventh embodiment of the present invention will be describedbelow by referring to the accompanying drawings.

FIG. 31 is a block diagram showing a configuration of thedriving-operation assist of the eleventh embodiment of the presentinvention. The driving-operation assist of this embodiment is alsomainly used to assist driving operations for garaging and parallelparking. Therefore, in this embodiment, it is assumed that a componentwith no explanation is the same as the case of the first and fifthembodiments and a component provided with the same symbol as the case ofthe first and fifth embodiments has the same function as the case of thefirst and fifth embodiments unless otherwise explained. Moreover, it isassumed that each modified example described for the first and fifthembodiments can be applied to this embodiment by applying the samemodification unless otherwise described.

The configuration of the driving-operation assist of this embodiment isdifferent from those of the first and fifth embodiments in that theobstacle inputting means 3101 and assumed-pattern correcting means 3102shown in FIG. 31 are included and the assumed-movement-pattern storingmeans 108 includes a movement pattern for changing backward and forwardmovements of a vehicle while the vehicle moves as shown in FIG. 32( a).

In this case, a steering-wheel angle corresponding to tire rotation isstored in the assumed-movement-pattern storing means 108 in FIG. 31 astime-series data for assumed-movement patterns as shown in FIG. 32( b).As shown in FIG. 32( b), tire rotation of 0 to 0.8 shows backwardmovement of a vehicle at which backward movement is changed to forwardmovement. In this case, the position of the vehicle is present at abackward- and forward-movement changing position 3201 shown in FIG. 32(a). Then, the vehicle moves forward to a tire angle of 0.8 to 0.6. Whenthe vehicle reaches the forward- and backward-movement changing position3202 shown in FIG. 32( a), forward movement is changed to backwardmovement again such as a tire angle of 0.6 to 1.4.

Thus, by including a movement pattern for changing backward and forwardmovements during movement, it is possible to control a position anddirection of a vehicle even if there is only a small spatial margin foran obstacle as shown in FIG. 32( a).

In this embodiment, a procedure until a circumferential-state image ofthe present invention is generated in accordance with the video dataimaged by the imaging section 101 is the same as that described for thefirst embodiment. A procedure until an assumed-movement pattern storedin the assumed-movement-pattern storing means 108 is displayed on asynthetic image of the present invention by adjusting an operation startposition of the pattern to the current position of a vehicle in aprocedure until the superimposing means 102 generates the syntheticimage and the displaying means 107 displays the synthetic image is thesame as that described for the first embodiment.

A procedure until a driver corrects an assumed-movement pattern andtime-series data by using the trace correcting means 1801 and obstacleinputting means 3101 after the assumed-movement pattern is displayed ona synthetic image and the corrected assumed-movement pattern andtime-series data are displayed on the synthetic image is described belowby using a case of performing garaging to the right side as an example.

As shown in FIG. 33, a case is assumed in which a driver moves a vehicleto the current position 1901 where an operation end position of anassumed-movement pattern coincides with the target parking position 1902in order to perform a garaging operation using the target parkingposition 1902 as an operation end position so that the vehicle does notcontact with an obstacle (a 3204, b 3205, or c 3206) and thereby, it isfound that the circumscribed area 604 of the assumed-movement patternmay contact with the obstacle a, b, or c.

When the assumed-movement-pattern storing means 108 stores anassumed-movement pattern for another garaging operation to the rightside, it is permitted to select the assumed-movement pattern by patternselecting means (not illustrated) and study if a parking operation canbe smoothly performed. However, when the means 108 does not store theassumed-movement pattern or another assumed-movement pattern alsointerferes with an obstacle or the like, the driver corrects theassumed-movement pattern.

First, the driver assumes a pattern showing the vehicle present at thecurrent position 3200 of the synthetic image (FIG. 33) shown on thedisplaying means 107 in FIG. 31 as the parking-operation start position1901, assumes an area in the image in which the obstacle a, b, or c ispresent as an obstacle designation rectangle: a 3207 as shown in FIG. 34by using the obstacle inputting means 3101 in FIG. 31 or as an obstacledesignation area: 3210 (refer to FIG. 36) by using an obstacledesignation circle: b 3208, and sets and inputs the obstacle designationrectangle or the obstacle designation area through numerical-valueinput, a pointer, or other means. Moreover, when it is necessary tocorrect the target parking position, the driver performs movement inputthrough numerical-value input, a pointer, or other means.

When an obstacle area is input, the trace correcting means 1801 sets acontact-hazard area 3209 in an area having a circumference of 60 cmincluding the obstacle designation area: 3210 as shown in FIG. 35.Moreover, a contact-hazard evaluation function H3210 as shown in FIG. 36is provided for the area. The function is configured by synthesizingthree quadratic functions which sharply increase as approaching for 10cm or less from the obstacle designation area: 3210, slowly decrease asseparating up to 10 cm or more, and become 0 for 60 cm or more.

Moreover, as shown in FIG. 37, an evaluation point 3211 (xi,yi):(i=1-6)is set to six points around a certain area as shown in FIG. 37 and atrace evaluation point 3212 (xi,yi)n:(n=1-N) about N items (tirerotation tm and tire angle km):(m=1-N) in the table shown in FIG. 32( b)is computed.

As shown by the expression in FIG. 38, a trace-contact-hazard evaluationfunction H″ 3213 is obtained from the total of the contact-hazardevaluation function H3210 at the position of the trace evaluation point3212.

The trace-contact-hazard evaluation function H″ 3213 becomes a functionof N items (tire rotation tm and tire angle km) in the table in FIG. 32(b) as shown in FIG. 37. Therefore, by successively correcting thediagram (tire rotation tm and tire angle km) through the partialdifferential method, it is possible to obtain values (tire rotation tmand tire angle km) for minimizing the trace-contact-hazard evaluationfunction H″ 3213.

Thereby, as shown in FIG. 39, it is possible to correct anassumed-movement pattern from (tire rotation tm and tire angle km) firstshown in FIG. 32( b) to (tire rotation tm and tire angle km) forminimizing the trace-contact-hazard evaluation function H″ 3213.

When even one point of 10 cm or less is found in the obstacledesignation area: 3210 from the contact-hazard evaluation function Habout each point of trace evaluation points 3212 (xi,yi)n in (tirerotation tm and tire angle km) for minimizing the trace-contact-hazardevaluation function H″ 3213, a warning Careful” is issued to a driver.When even one point of 0 cm or less is found in the obstacle designationarea: 3210, a warning Parking not possible” is issued to a driver.

Because the corrected assumed-movement pattern is a movement pattern forminimizing the trace-contact-hazard evaluation function H″ 3213, theassumed-movement pattern generates a trace having a larger margin froman obstacle as shown in FIG. 40 and thereby, a vehicle can be parkedmore safely.

As shown in FIG. 41, the superimposing means 102 generates a syntheticimage by adjusting the operation start position 1901 of a correctedassumed-movement pattern 3214 to a current vehicle position 3200 and thedisplaying means 107 displays the synthetic image.

Therefore, when a driver starts a driving operation (parking operation)of a vehicle in accordance with the new corrected assumed-movementpattern 3214, the driver can park the vehicle at the target parkingposition 1902 in accordance with a movement pattern having a largermargin from an obstacle.

It is permitted to store generated new assumed-movement pattern andtime-series data in the assumed-movement-pattern storing means 108 byupdating an original assumed-movement pattern or to additionally storethe new assumed-movement pattern in the means 108 without changing theoriginal assumed-movement pattern. Moreover, it is permitted totemporarily use them without storing them. Furthermore, it is permittedthat a driver selects renewed storage, additional storage, ornon-storage each time.

Furthermore, this embodiment is described above by assuming that anassumed-movement pattern to be renewedly-stored or additionally-storedin the assumed-movement-pattern storing means 108 can be automaticallyobtained in accordance with a position of a vehicle at start or end of amovement input from a driver. However, it is also permitted to performan actual driving operation and thereby, sample time-series data for thethen steering angle of a steering wheel and the number of revolutions ofwheels, generate an assumed-movement pattern in accordance with thetime-series data, and store the pattern.

Twelfth Embodiment

Then, a twelfth embodiment of the present invention will be describedbelow by referring to FIGS. 42 to 46.

In the eleventh embodiment, it is assumed that an assumed-movementpattern shown in FIG. 32( a) is corrected in accordance with an obstaclearea input from a driver. However, it is also permitted to correct twoassumed-movement patterns or more and select a preferable one. FIG. 42is different from FIG. 31 in that movement-pattern selection tracecorrecting means 3301 is used instead of the trace correcting means3101.

First, operations of the twelfth embodiment are described. As shown inFIG. 43, a driver designates and inputs the target parking position 1902on an image displayed on a display unit by assuming a current vehicleposition as a parking-operation start position. The movement-patternselection trace correcting means 3301 extracts two assumed-movementpatterns for parking a vehicle right backward as shown in FIG. 44 inaccordance with a rough positional relation of the target parkingposition 1902 to the parking-operation start position out of a pluralityof assumed-movement patterns stored in the assumed-movement-patternstoring means 108.

Moreover, as shown in FIGS. 45 and 46, when the driver inputs anobstacle area on the image displayed on the display unit, the twoassumed-movement patterns for respectively minimizing thetrace-contact-hazard evaluation function H″ 3213 are corrected similarlyto the case of the eleventh embodiment. In this case, by comparing thetwo minimized trace-contact-hazard evaluation functions H″ 3213 witheach other and selecting a smaller one of the two functions H″ 3213, itis possible to select a safer assumed-movement pattern. Moreover, whenthe two trace-contact-hazard evaluation functions H″ 3213 have the samevalue, a simpler assumed-movement pattern is selected by previouslyraising the priority of a function H″ 3213 allowing a simpler drivingoperation.

Therefore, when a spatial margin from an obstacle is small, a safermovement pattern having a cutback is automatically selected as shown inFIG. 45. However, when a spatial margin from an obstacle is large, asimpler movement pattern having no cutback is automatically selected asshown in FIG. 446 [sic; FIG. 46].

According to the above configuration, a driver can automatically selectan optimum assumed-movement pattern only by inputting a target parkingposition and an obstacle area and thereby realize optimum parkingthrough a safer and simpler driving operation.

According to this embodiment, it is possible to realize an extensibledriving-operation assist compared to the driving-operation assist of thefirst embodiment.

The above first to twelfth embodiments are described by assuming thatcircumferential-state imaging means of the present invention mainlygenerates an image viewed from a viewpoint of a virtual camera by usinga plurality of vehicle-mounted cameras. However, it is also permitted touse one camera set to the ceiling of a roof-provided parking lot. Inshort, it is possible to use any type of means as circumferential-stateimaging means of the present invention as long as the means generates acircumferential-state image by imaging a circumferential state of avehicle with a camera and/or stores the generated circumferential-stateimage.

Moreover, the above first to twelfth embodiments are described byassuming that driving-operation assist of the embodiments arerespectively provided with assumed-movement-pattern storing means of thepresent invention. However, it is also permitted to use any type ofmeans as assumed-movement-pattern storing means of the present inventionas long as an assumed-movement pattern can be input to the means eachtime. In short, it is permitted to use any type of driving-operationassist as driving-operation assist of the present invention as long asthe driving-operation assist includes circumferential-state imagingmeans for imaging a circumferential state of a vehicle with a camera andthereby generating a circumferential-state image and/or storing thecircumferential-state image, synthetic-image generating means forsuperimposing on the circumferential-state image, an assumed-movementpattern which is video data showing movement of the vehicle in the caseof performing a predetermined series of driving operations for thevehicle and generating a synthetic image, and displaying means fordisplaying the synthetic image. Thereby, because at least acircumferential-state image and an assumed-movement pattern aresuperimposed on each other and displayed on a synthetic screen, a drivercan confirm the relation between the circumferential-state image and theassumed-movement pattern at a glance.

INDUSTRIAL APPLICABILITY

As apparent from the above description, the present invention provides adriving-operation assist by which a driver can directly confirm therelation between vehicle movement according to a predetermined series ofdriving operations and a circumferential state by displaying the vehiclemovement when the driver performs the predetermined series of drivingoperations together with the circumferential sate and a driver's loadcan be reduced.

That is, a driver can confirm a position for starting a drivingoperation such as garaging or parallel parking, a place for finallystopping a vehicle, and a positional relation with an obstacle such asother vehicle by the display image at a glance by using adriving-operation assist of the present invention. Therefore, it isexpected that a driver's operation load is reduced and safety isimproved. Moreover, by introducing an automatic-driving technique, it ispossible to automatically perform every operation including garagingonly by driver's moving a vehicle up to a parking-operation startposition while viewing the movement trace data.

Furthermore, the present invention provides a recording medium forstoring a program for making a computer execute all or some of functionsof each means of a driving-operation assist of the present invention.

1. A vehicle operation assist for a vehicle mounted with a camera, saidvehicle operation assist comprising: an imaging unit operable togenerate a circumferential state image using an output image from thecamera, said circumferential state image viewed from a virtualviewpoint, said virtual viewpoint being different from a viewpoint ofthe camera; a synthetic-image generating unit operable to generate asynthetic image by using the circumferential state image and anassumed-movement pattern image which is based on a predetermined drivingoperation of the vehicle, the assumed-movement pattern image including amovement of the vehicle between a start position of the vehicle and aparking position of the vehicle in case of performing the predetermineddriving operation; and a display unit operable to display the syntheticimage, wherein the assumed-movement pattern image includes tire traces.2. A method of vehicle operation assist comprising the steps of: (a)imaging a circumferential state of a vehicle; (b) generating acircumferential-state image of the vehicle as viewed from a virtualviewpoint based on the imaging of step (a); (c) generating a syntheticimage of the vehicle by using the circumferential state image of thevehicle with respect to an assumed-movement pattern image which isobtained from a series of a predetermined driving operation for thevehicle, the assumed-movement pattern image including a movement of thevehicle between a start position of the vehicle and a parking positionof the vehicle in case of performing the predetermined drivingoperation; and (d) displaying the synthetic image of the vehicle,wherein the assumed-movement pattern image includes tire traces.